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1 METHODS AND APPARATUSES FOR DETERMINING THE ORIENTATION OF 

2 AN OBJECT IN AN IMAGE 

3 This application claims the benefit of U. S. Provisional Application Number 

4 60/147,721, filed August 6, 1999. 

5 FIELD OF THE INVENTION 

6 This invention relates to machine vision, and particularly to methods and 

7 apparatuses for processing images, 

8 BACKGROUND 

9 Machine vision refers to the automated analysis of an image to determine 



10 characteristics of objects and other features shown therein. It is often employed in 

J 1 1 automated manufacturing lines, where images of components are analyzed to determine 

if : 12 placement and alignment before assembly. Machine vision is also used for quality 

%0 13 assurance. For example, in the pharmaceutical and food packing industries, images of 

g 14 packages are analyzed to insure that product labels, lot numbers, "freshness" dates, and 

m 15 the like, are properly positioned and legible. 

O 16 In many machine vision applications, it is essential to in determining the position 

s 5 1 17 and orientation of an imaged object. In electronic circuit board assembly, for example, 

^ 18 integrated circuit chips must be precisely positioned before they can be soldered into 

5"*""! 

□ 19 place. Metallic solder pads feature predominantly in images of these components and, as 

20 such are often used in determining chip location and orientation. For example, a vision- 

21 based automatic surface mounter (SMD) machine typically employs a geometrical-based 

22 model of leaded objects to accurately inspect and place leaded objects on a printed circuit 

23 board (PCB). A leaded object 900 is an electronic component that has an object body 

24 902 and leads 904, an example of which is depicted in Fig. 9, which is discussed more 

25 fully with reference to Fig. 8 hereinafter. The SMD machine places the leaded objects by 

26 registering the leaded object 900 with pads on the PCB, such that the centers of feet 906 

27 of the leaded object 900 align with the centers of pads on the PCB within positional 

28 tolerances. 
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Objects whose image is to be analyzed by machine vision typically include 
calibration targets. Often a cross-shaped symbol or parallel array of dots, such a target 
facilitates determining the position and orientation of the object in the image. 

If the object is composed of lines, such as the calibration target, there are several 
known approaches to the problem of finding the angle of a line in an image. One of the 
best known techniques is the Hough line-transform. The Hough line-transform 
transforms characteristics of edge points from image into line characteristics, and 
represents the line characteristics on a Hough space, where a point in the Hough space 
represents a line. Though often used, the Hough transform can require significant 
computation resources and time. Another technique to find lines is described in Machine 
Vision Method for Identifying Collinear sets of Points from an Image, U.S. Patent 
Number 6,075,88 1 The '88 1 patent employs an angle-finding phase and a line-finding 
phase to find the lines. During the angle-finding phase, the angular orientation of a 
predominant line is estimated by projecting data points at multiple angles, (i.e., summed 
in the direction of the projection, where projection is a term known in the art and 
described at B.K.D. Horn, "Robot Vision," MIT Press, London England, 1985, Section 
3.3 pp. 53 - 57). An angle of the predominant line is used by the line finding-phase to 
find lines by identifying sets of data points aligned with the angle. The line finding phase 
projects the data points at the angle, and the lines are located at peaks in the projections, 
as is further described in the '881 patent. Both of these methods, however, are directed at 
finding lines, and may not work in all applications, such as for short lines or noisy 
images, for example. Short lines are difficult to identify because not enough edges 
accumulate in a Hough bin or not enough data points cluster in the projection, for 
example. Further, it is difficult to identify lines from noisy images containing the lines, 
such as images of specularly reflecting leads, where specularity is known in the art and is 
described further hereinafter. The noise masks the lines, thus, making it difficult to 
identify the lines amidst a noisy Hough space or a noisy projection, for example. 

Another known method uses the angle of strong edge features to determine the 
orientation. Typically, edge detection is performed on the image, and the results are 



1 stored in an edge image. The edges within the edge image are projected at various 

2 angles, such as through a Radon transform, for example. Peaks in the Radon transform 

3 indicate the possible presence of lines that correspond to the angles of the image or 

4 objects in the image that have prominent edge patterns. Angles of images and some 

5 objects can be determined from projecting edges, but the method is also susceptible to be 

6 misled by stray edges and noise. 

7 SUMMARY 

8 This invention provides methods and apparatuses for determining an orientation 

9 of an object from an image of the object. First, a two-dimensional frequency response of 

10 the image, or portion thereof, is determined, and, preferably, represented in a frequency 

1 1 space, such as a two-dimensional plot or a frequency image, for example. Second, the 

12 frequency space is evaluated to determine an orientation of a spatial-frequency pattern 

13 therein. The invention recognizes, among other things, that the orientation of the spatial- 

14 frequency pattern within the frequency space is related to the orientation of the object in 

15 the image. More generally, the invention recognizes the frequency response of an image, 

16 or portion thereof, contains information about the orientation of objects in the image. 



17 In a preferred embodiment, the invention recognizes that a magnitude of a 

is frequency response of an image, or portion thereof, contains a spatial-frequency pattern 

19 or patterns, and that the orientation of the spatial-frequency pattern or patterns is related 

20 to, and often the equivalent of, the orientation of the object in the image. 

21 Further, the invention recognizes that once the frequency response, or portion 

22 thereof, is represented two dimensionally in a frequency space, whether plotted or within 

23 a frequency image, for example, and the frequency response, or portion thereof is 

24 preferably scaled to enhance the high frequencies, the orientation information is easily 

25 obtained from the frequency space using a variety of algorithms. 

26 In one embodiment, the frequency response, or portion thereof, is scaled using a 

27 logarithm scaling function. 

28 In a preferred embodiment, the frequency response, or portion thereof, determined 

29 is a magnitude of a two-dimensional discrete fast Fourier transform ( W 2-D FFT"). The 

3 




1 spatial-frequency pattern or patterns formed in the frequency space from the scaled 

2 magnitude of the 2-D FFT include a line, lines, and/or at least one set of substantially 

3 parallel lines. An angle of the line, denoted line angle, substantially equals, or is at a 

4 constant offset from the angle of the object in the image, where the line angle can include 

5 the angle of features on an object and/or the angle of the object itself. 

6 The orientation of the object is optionally used in subsequent image processing, 

7 such as searching, gauging, inspecting, and, in a preferred embodiment, modeling the 

8 features of object in the image. 

9 One of the advantages of the invention is that it is efficient in time. A further 



10 advantage is that the invention can robustly determine the angle of an object from an 

i l image, regardless of extraneous features in the image, such as artifacts and noise, for 

J 12 example, when the object produces a high frequency response. The extraneous features 

J 13 do not mask the frequency response, and therefore, do not interfere with identification of 

vjj 14 the angle of the object Further, the invention is more robust than other methods that 

I 1 5 cannot distinguish extraneous features juxtaposed and/or surrounding the object, such as 

^ 16 projections, for example. 

O 17 hi further aspects, the invention presents methods and apparatuses to determine 

h 5 *5 1 8 models, gauge, inspect, or locate features in images. 

^19 In further aspects, the invention provides an apparatus in accord with the methods 

p 20 described above. The aforementioned and other aspects of the invention are evident in 

21 the drawings and in the description that follows. 

22 BRIEF DESCRIPTION OF DRAWINGS 

23 The invention will be more fully understood from the following detailed 

24 description, in conjunction with the accompanying figures, wherein: 

25 Fig. 1 depicts a schematic machine vision system for practice of the invention; 

26 Fig. 2 is a flowchart summarizing operation of an embodiment of a method 

27 according to the invention that determines the angle of at least one object and/or feature 

28 in an image; 
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1 Fig. 3 depicts the operation of applying a frequency analysis of Fig. 2 to two 

2 synthetically generated images; 

3 Fig. 4 depicts the operation of Fig. 2 applied to two synthetically generated 

4 images; 

5 Figs, 5 A, 5B, 5C, 5D, 5E, and 5F depict images of objects and/or feature(s), not 

6 drawn to scale, and a representation of the absolute value of a frequency response, or 

7 portion thereof, of the objects and/or feature(s); 

8 Fig. 6 depicts objects and features, not drawn to scale, whose angle can also be 

9 determined using the method of the invention; 

10 Fig. 7 depicts an image of object containing writing that can be evaluated using 

11 Fig. 2; 

12 Fig, 8 is a flowchart summarizing operation of an embodiment of a method 

13 according to the invention that generates a model; 

14 Figs. 9, 9 A, 9B, 9C, and 9D depict a leaded object, not drawn to scale, an image 

1 5 of the leaded object, an image of a portion of a lead set the leaded object, a histogram 

16 representing edges of the lead set, and a portion of an edge list derived from the 

17 histogram, respectively, where the leaded object and be modeled in whole, or in part, 

18 using the invention; 

19 Figs. 10A and 10B are flowcharts summarizing operation of an embodiment of a 

20 method according to the invention that uses the angle of at least one feature in an image 

21 to evaluate the image; and 

22 Fig. 1 1 depicts features and objects, not drawn to scale, that can be modeled using 

23 the method of the invention. 

24 DETAILED DESCRIPTION OF DRAWINGS 

25 The methods and apparatuses determine the orientation of at least one feature in 

26 an image. The method is particularly useful for locating the angle of a lead set of a 

27 leaded object, which, optionally, is then used in further aspects to generate a geometric 

28 model of the lead set and, optionally, the leaded object Though this is the form of a 
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1 preferred embodiment, this embodiment should be considered illustrative, and not 

2 restrictive. 

3 Fig. 1 illustrates a machine system 1 1 0 of the type in which the invention is 

4 practiced. The system 1 10 includes a capturing device 1 16, such as a conventional video 

5 camera or scanner, that generates an image of an object 1 12. Image data (or pixels) 

6 generated by the capturing device 1 16 represent, in the conventional manner, the image 

7 intensity (e.g. color or brightness) of each point in the scene at the resolution of the 

8 capturing device 116. 

9 The capturing device 1 1 6 transmits image data via a communications path 1 1 8 to 

10 an image analysis system 120. This is a conventional digital data processor, or a vision 

1 1 processing system of the type commercially available, for example, from the assignee 
O 12 hereof, Cognex Corporation, programmed in accord with the teachings hereof to 

p 13 determine the angle of an object and/or feature in an image from the image data, and, 

JJ{ 14 optionally, generate a geometric model of the object and/or feature 1 12. 
W 15 The image analysis system 120 may have one or more central processing units 

III 16 122, main memory 124, an input-output system 126, and one or more disk drives (or 

JL, 1 7 other mass storage device) 128, all of the conventional type. 

^ 18 Tfl e system 120 and, more particularly, central processing unit 122, is configured 

P 1 9 b Y programming instructions according to the teaching hereof to provide the angle of the 

« 20 object and/or features in an image from the image data. The central processing unit 122 

21 includes a frequency device 134, an orientation device 136, and, optionally, an image 

22 evaluation device 144. The frequency device 134 and the orientation device 136 generate 

23 a frequency response of an image, or portion thereof, and evaluate an angle of a spatial 

24 pattern(s), also called spatial-frequency patterns), present therein to provide the angle of 

25 the object and/or features in the image from the image data, as described hereinafter. The 

26 angle is optionally input into the image evaluation device 144, and used therein during 

27 subsequent image processing, such as searching, gauging, inspecting, and/or modeling of 

28 the object and/or features in the image, for example. 
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1 It should be apparent to one skilled in the art that devices within the central 

2 processing unit 122 can be partitioned in more than one manner without departing from 

3 the scope of the invention. 

4 Those skilled in the art will appreciate that, in addition to implementation on a 

5 programmable digital data processor, the methods and apparatuses taught herein can be 

6 implemented in special purpose hardware. 

7 Fig. 2 illustrates a flowchart of an embodiment of a method according to the 

8 invention, where steps of the method will be denoted in the detailed description in 

v parentheses. The first step is to acquire an image of the object (200) Next, a frequency 

io response of the image, or portion thereof, is generated (202) Thereafter, the frequency 

1 l response, or portion thereof, is analyzed to determine the angle of an object and/or 

12 features in the image (204). 

13 Tne frequency response is analyzed because, as recognized by the invention, the 

14 frequency response, or portion thereof, contains information about the orientation of 

15 objects in the image. More specifically, the invention recognizes that the frequency 

16 response, or portion thereof, contains a spatial pattern or patterns that is, or are, related to, 

1 7 and often the equivalent of, the angle of the object in the image. 

*8 Examples of specific frequency responses of images 300 and 310 of synthetical }y 

19 generated striped objects, herein after termed idealized object 320 are shown in Fig. 3, 

20 where the specific frequency responses 302 and 3 1 2, are a magnitude of the two- 

21 dimensional discrete Fourier transform ("2-D DFT") of the images 300 and 310. The 

22 frequency responses 302 and 3 1 2 are represented on a dual-axis frequency space 304 and 

23 314, respectively. 

24 The 2-D DFT is given by the following equations 

<N,-1) <N 2 ~1) */*>tt/ "VI \L 

25 X(k,,k 2 )= Z Z x( ni ,n 2 )e' j(2TT/N,)kini e* 2 ™^**"* 

ni ^ 0 n 2 ^ 0 

26 for 0 < k, < N, - 1 and 0 < k 2 < N 2 1 
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1 X(k 1? k 2 ) = 0 for all other k, and k 2 . 

2 X(ki,k 2 ) is the frequency response of the image, ki and k 2 are integers, and 

3 X(k],k 2 ), at each ki and k 2 , is the value of the frequency response in a frequency space 

4 having KI and K2 as frequency axes. The image is a discrete, two-dimensional function, 

5 given by x(ni,n 2 ), where ni and n 2 are integers, and the value ofx(n b n 2 ) at each point 

6 (i.e., each ni and n 2 ) represents a shade of grey in the image. The image size is Nj by N 2; 

7 where Nj and N 2 are positive integers. The image has an y-axis and an x-axis. 

8 The Fourier frequency analysis represents an image, x(n i? n 2 ), of size Ni by N 2 , 

9 (i.e., an Ni x N 2 point sequence), in the frequency domain by X(ki,k 2 ), which is an Ni x 

10 N 2 point sequence. 

1 1 Returning to Fig. 3, image 300, depicts the idealized striped object 320 positioned 

12 in the image 300 so that the dark stripes 326 extend horizontally across the image 300. 

13 At a single y- value, the image 300 does not change grey level along the x-axis. The 

14 frequency response, X(ki,k 2 ), 302, shown on the frequency space 304, reflects the lack of 

15 change in grey level of the striped object 320 along the x-axis. The x-axis of the image 

16 300 contributes only to the EXT component of the frequency response. Thus, frequencies 

17 are only present at the DC component along the Kl-axis, specifically frequencies are 

18 present where ki = the DC component. Frequencies exist along the K2-axis and are 

19 shown in plot 306, which is a cross-section of the frequency space 304 at AA\ The plot 

20 306 depicts the value of the frequency along the abscissa and the magnitude of the 

21 frequency, X M , along the ordinate. 

22 As known in the art, X(ki,k 2 ) is often complex and is expressed by a magnitude 

23 component, X M , and a phase component, 9 X , or a real component, X r , and a complex 

24 component, Xi. 

25 In a preferred embodiment, the frequency response of the image that is generated 

26 is the magnitude, X M , where X M = |X(ki,k 2 )| = sqrt(X r 2 + X, 2 ). X M is a measure of the 

27 strength of the spatial frequencies present in the image, and the strength is represented by 

28 the height of the peaks 308 in the plot 306, and is also represented by the brightness or 
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1 darkness of the features in the frequency images described hereinafter. Peaks, as used 

2 herein, exhibit higher magnitudes of frequencies relative to the background level. 

3 In contrast to image 300, image 310 depicts the same idealized striped object 320 

4 positioned differently, so that the dark stripes 326 extend vertically across the image 310. 

5 At a single x-value, the image 310 does not change grey level along the y-axis. 

6 Accordingly, the frequency response, Xik u k 2 \ 3 12 is only present at one point along the 

7 K2-axis, specifically, frequencies are present where k2 = the DC component, while 

8 multiple frequencies exists along the Kl-axis and are shown in plot 3 16, which is a cross- 

9 section of the frequency space 314 at BET. The plot 316 depicts the value of the 

10 frequency along the abscissa and the magnitude of the frequency, X M , along the ordinate, 

1 1 Fig. 4 depicts two images 402 and 452 of the same striped object 320 positioned 

12 to provide changes in grey values in both x- and y- directions within the images 402 and 

13 452. Accordingly, there are components present in both Kl- and K2- directions within 

14 the frequency responses 406 and 456 as depicted in the frequency spaces 404 and 454 of 

1 5 the images 402 and 452, respectively. 

16 As the striped object 320 moves, the frequency response changes, and the peaks 

17 within the frequency response move. For example, the position of the peaks 400, in the 

18 frequency response 406 are displaced from the peaks 400 in the frequency response 456. 

19 Specifically, the frequency response 406, to the object positioned at 45°, has peaks 400 

20 that are generally displaced to the right of, and higher than, the peaks 400 in the 

21 frequency response 456, to the object positioned at 20°. 

22 The position of the peaks 400 will move relatively as the striped object 320 

23 changes direction, because when striped object 320 moves, the 2-D DFT responds more 

24 strongly to a different set of frequencies than the 2-D DFT responded to before the 

25 striped object 320 moved. In general, frequency is a measure of change in space or in 

26 time. Spatial frequencies of an image are the measure of the rate of change in grey level 

27 across an image. Such grey level changes are typically caused by edges. When the edges 

28 of the objects move, the frequency response of the image changes accordingly. 
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1 The same striped object 320, positioned four ways, in four images 302, 3 1 0, 402, 

2 and 452, generated four different frequency responses 302, 3 12, 406, and 456, 

3 respectively. The frequency response 302 of the image 300, wherein the stripes 326 are 

4 horizontal, is a vertical line 322. The frequency response 3 12 of the image 3 10, wherein 

5 the stripes 326 are vertical, is a horizontal line 324. The frequency response 406 of the 

6 image 402, wherein the stripes 326 are positioned at 45°, appeared on the frequency 

7 space 404 as two sets of parallel lines 408 and 410, one set of parallel lines 408 

8 positioned at 45° and one set of parallel lines 410 positioned at 135°. The frequency 

9 response 456 of the image 452, wherein the stripes 326 are positioned at 20°, appeared on 
1 o the frequency space 454 as two sets of somewhat parallel lines 458 and 460, one set 458 

0 1 1 positioned at approximately 20° and one set 460 positioned at approximately 160°. 

m 12 All the frequency responses 302, 312, 406, and 456 are related to the angle of the 

W 13 idealized object 320 and the angle of the striped features 326 in the images 300, 310, 402, 

B 14 and 452, respectively. More particularly, the lines, such as 322, 324, 408, 410, 458 and 

Jt 15 460 (i.e., the spatial patterns), of the frequency responses 302, 3 12, 406, and 456 share, or 

1 16 are at a constant offset from, the angle of the idealized object 320 and/or the stripes 326. 
si 17 For instance, for the frequency response 406, the two sets of parallel lines 408 and 410 
rf is reflect the sides, and ends, of the stripes 326 (i.e. features), respectively. Therefore, the 
Q 19 angle of the set of parallel lines 408 in the frequency space 404 is substantially equivalent 

° 20 to the angle of the width, w, of the stripes 326 in the image 402, and the angle of the set 

21 of parallel lines 410 in the frequency space 404 is substantially equivalent to the angle for 

22 the length, /, of the stripes 326 in the image. 

23 The magnitude of the frequencies, and thus, that of the peaks 400 can be 

24 depicted in numerous manners, including by graphing the frequency response along a 

25 cross section of the frequency spaces 404 and 454, similarly to plots 306 and 316 of the 

26 horizontal and vertical idealized objects 320. 

27 The invention recognizes that the magnitude of the frequency response, or portion 

28 thereof, of an image, contains a spatial pattern or patterns, and that the angle of the spatial 

29 pattern or patterns is related to, and often the equivalent of, the angle of the object in the 

10 



1 image. The angle of the spatial pattern is related to the object by being at a constant 

2 offset from the angle of the object and/or features on the object 

3 Figs. 3 and 4 each depict one instance of the frequency response, or portion 

4 thereof, of the same idealized object 320 positioned at a given angle. 

5 However, in manufacturing conditions, more than one instance of a frequency 

6 response to the same object is possible. This effect can be illustrated, in part, by Figs. 5A 

7 and 5B, not drawn to scale, which depict two images 500 and 5 10 of a pair of lead sets, 

8 positioned within each image 500 and 5 10 at the same angle, and representations of the 

9 corresponding frequency responses 502 and 5 12. In image 510, each lead 5 16 has three 

10 regions, 524, 526, and 528. The presence of three regions is a typical effect caused by 

1 1 the specularly reflecting surface of the leads 516, where the specular reflections cause 

'% 12 images of leads to display variations in brightness levels. Specular reflections from leads 

E 1 1 3 and the effects on images are well known in the art. For illustration purposes, the image 

C! 14 510 of the lead sets 514 is idealized so that each of the leads 516 is shown exhibiting the 

15 same three regions 524, 526, and 528. The second idealized image, image 500 of the lead 

U1 16 sets 5 15 shows an image 500 that is not effected by specular reflection, so that each of the 

□ 17 leads 5 17 images as one region. Although both images 500 and 510 are of two 6-leaded 

18 lead sets 514 and 515, positioned at the same angle, the frequency responses 502 mid 512 

^ 19 are different. The frequency response 512 of the image 510, containing the specularly 

O 20 reflecting leads 516, is one set of parallel lines 518, while the frequency response 502, of 

21 the image 500, of the non-specularly reflecting leads 5 17, is a set of parallel lines 522 and 

22 a line 520 positioned orthogonal thereto. 

23 Additionally, a spatial pattern within a frequency response can be due to factors 

24 other than the object or its features, such as the windowing of the image, where 

25 windowing is a term known in the art. For example, Fig. 5C depicts an image of a circuit 

26 board 530, not drawn to scale, and a representation of a corresponding frequency 

27 response, or portion thereof, 532. The frequency response 532 contains eight lines 

28 sharing four angles, two sets of parallel lines 534 and 536 are related to the angle of the 

29 features on the circuit board, and two other lines 538 and 540 are related to the 

ll 



1 windowing of the image 530. The lines 538 and 540 are part of two sets of parallel lines 

2 (not shown) sharing the angles of 0° and 90°. The other parallel lines had less intensity 

3 and were omitted for illustration purposes. 

4 Further still, the frequency response can be a single line at each angle, as opposed 

5 to a set, or sets, of parallel lines. For instance, Fig. 5D illustrates an image 550 of a top- 

6 view of a portion of an integrated circuit board, not drawn to scale, and a representation 

7 of the corresponding frequency response, or portion thereof 552. The integrated circuit 

8 board has several features, or portions thereof, aligned at approximately 30° and 120°. 

9 The frequency response 552 is two single lines 554 and 556, orthogonal to one another, 

10 as opposed to one or more sets of parallel lines. Returning to Fig. 3 previously described, 
OH the frequency responses 302 and 3 12 are other examples where the spatial pattern is a 

■K 1 2 single line 322 and 324, respectively. 

f y 1 3 Another example of a frequency response is the discrete circles shown at the 

m 14 intersections of the lines in Fig. 5E, which depicts an image of a ball grid array and a 

t; 1 5 representation of a corresponding frequency response, or portion thereof 

a 16 In one embodiment, to decrease processing requirements, only part of the 

Si 17 frequency response is calculated and/or evaluated in step (204). 

*fj i g Also to decrease processing time, in a preferred embodiment, a two-dimensional 

O 19 discrete fast Fourier Transform of the image is computed, using techniques known in the 

20 art. 

21 It should be apparent to those skilled in the art, that other discrete signal analysis 

22 techniques, such as z-transforms and cosine transforms, can be used in accordance with 

23 the teachings herein without departing from the scope of the invention. One skilled in the 

24 art, using the teachings herein, can easily verify which frequency analysis tools are 

25 useful, and the list given here is not inclusive. 

26 Further, although the two-dimensional analysis described herein was in the x- and 

27 y- direction, other spatial representations can also be used, such as radial and 

28 circumference directions of r and 0 without departing from the scope of the invention. 
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1 Further still, the frequency analysis can be applied to more than two dimensions. 

2 For instance, a three-dimensional image signal (i.e. a signal that is a function of x, y, and 

3 z, for example) can be evaluated using the teachings herein, such as applying a three- 

4 dimensional discrete Fourier Transform. In this way, frequencies along the z-direction 

5 are examined. The three-dimensional image signal is constructed as known in the art; 

6 such as by building a three-dimensional object representation from two-dimensional 

7 slices of the object, for example. 



8 Once the frequency response, or portion thereof, is generated, the spatial pattern 

9 within the frequency response, or portion thereof, is evaluated for the orientation 

10 information (204). 

1 1 Returning to Fig. 4, for illustration purposes, the frequency response, or portion 

12 thereof 406 and 456, is represented graphically within a two-dimensional frequency space 

13 404 and 454, respectively. The frequency spaces 404 and 454 contain plots of values of 

14 the frequency responses or grey-scale images representing the frequency responses 406 

1 5 and 456, respectively. 

16 When the frequency response is represented within an image, the frequency 



17 response is mapped to a grey-scale value in the image between 0 and 256. It is preferred 

18 that the grey values in the frequency image are mapped so that the low grey values 

19 represent low magnitudes in the frequency response and high grey values represent high 

20 magnitudes in the frequency response. It should be apparent that other mappings can be 

21 used, such as using the period, as opposed to the frequency, or mapping the high 

22 magnitudes using the low grey values, and the low magnitudes using the high grey 

23 values, for example. For illustration purposes, the figures used herein map the high 

24 magnitudes in the frequency response using the low grey values and the low magnitudes 

25 in the frequency response using the high grey values, so that the peaks of the frequencies 

26 400 appear as darker features within the frequency images. 

27 In a preferred embodiment, before the frequency space is evaluated, the stronger 

28 DC peak and other low frequency peaks, such as peak 422 on Fig. 4 or peak 320 on plot 

29 3 16 of Fig. 3, within the frequency response, or portion thereof, are minimized and/or the 
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1 higher frequencies are enhanced using a scaling factor. Preferably, the frequency 

2 response, X(kl ? k2), is scaled using a logarithmic function, such that the frequency image 

3 is mapped from the function F(x,yX where F(x, y) = log |X(kl, k2)|, and X(kl, k2) as 

4 given above. The logarithmic function decreases the effect of the stranger DC peak and 

5 other low frequency peaks, and, thus, allows easier analysis of the smaller peaks. It 

6 should be apparent to one skilled in the art that other scaling functions can be used, such 

7 as raising X(kl, k2) to a fraction of a power, for example. 

8 Optionally, the frequency response is filtered to minimize and/or remove 

9 extraneous data before the frequency response is evaluated Frequency response 412 is 

10 an example of an instance of the frequency response 406 after being scaled and filtered, 

1 1 and frequency response 462 is an example of an instance of the frequency response 456 

12 after being scaled and filtered. The filtering is accomplished by one of any number of 

13 vehicles known in the art. For instance, only the data points above a value threshold are 

14 retained for a plot or only data points above a grey value threshold are retained for an 

1 5 image. Alternatively, an edge operator is applied to a frequency space and the data points 

16 retained are only edges above a contrast threshold, or only "true peak" edges, for 

1 7 example, where edge operators and "true peaks" are terms and techniques known in the 

18 art. 

19 The invention recognizes that a variety of algorithms can determine the angle of 

20 the spatial pattem(s) once the frequency response, or portion thereof, is represented two 

21 dimensionally in a frequency space, whether plotted or within a frequency image, for 

22 example, and the frequency response, or portion thereof, is preferably scaled and, 

23 optionally, filtered to enhance the high frequencies. 

24 A variety of algorithms can analyze the frequency response, or portion thereof, to 

25 find the angle of the spatial pattern(s). For example, the angle of pattern(s) is found using 

26 projections. With this technique, the frequency image is projected at a plurality of angles 

27 to create a one-dimensional ("1-D") array for each of the angles, where 1-D projections 

28 are a technique known in the art, and further described in Cognex 300/400/500, Image 

29 Processing, Programmable Vision Engines, Revision 7,4 (1996) pp. 343 ~ 347. The 1-D 
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1 arrays are evaluated far peaks, and the angle(s) of the? projections producing higher 

2 peak(s) are the dominant angles) of the features in the image. For example, Fig. 4 

3 depicts an instance of a 1-D array 426 formed from the projection of the frequency image 

4 412 at 135°. Projecting the frequency image 412 at 135° produced high peaks, such as 

5 peaks 427. Thus, 135° is a dominant angle of the features (i.e., the lines) in the image 

6 412. Also shown is an instance of a 1-D array 428 formed from the projection of the 

7 frequency image 462 at 160°. Projecting the frequency image 462 at 160° produced high 

8 peaks, such as peaks 429. Thus, 160° is a dominant angle of the features (i.e., the lines) 

9 in the frequency image 462. Those skilled in the art will appreciate that the peaks 427 

10 and 429 represent portions of the 1-D array that exhibit a higher density of projected-grey 

1 1 levels relative to the background level thereof Those skilled in the art will also realize 

12 that a different measure for the dominant angle could be used, such as the angle that 

13 produced the highest peak, the most high peaks, or other similar measure without 

14 departing from the scope of the invention. 

15 Although empirically the patterns found have been lines or discrete circles, such 

16 as the discrete circles of Fig. 5E, other objects could generate other patterns within the 

17 frequency space. For such patterns, either the spatial pattern would need to be identified 

18 in order to choose the algorithm to evaluate it, or an algorithm that is not dependent on 

19 shape would be used, such as projection for certain objects. 

20 An additional method for identifying lines, and the angle of the lines, from images 

2 1 has been described in commonly assigned, U.S. Patent Number 6,075,88 1 , incorporated 

22 herein by reference. 

23 A still further method for identifying lines and the angle of the lines is applying a 

24 Hough-line finding algorithm. A Hough space for lines is a two-dimensional space in 

25 which lines from an image are recorded. The Hough space is adequately represented as a 

26 two-dimensional array of bins 446. One axis 442 represents the line-angle, a, and the 

27 other axis 444 represents the distance vector, d, which is the shortest distance from the 

28 origin of a coordinate system to the line. A point on a Hough space represents a line. 

29 Every data point from the frequency image "votes" into an accumulator array in the 
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1 Hough space. The bins 446 with sufficiently many votes are identified as lines. For 

2 example the lines 414, 416, 418, and 420 are identified as peak bins 434, 436, 438, and 

3 440, respectively, in a Hough space 432, and lines 464, 466, 468, and 470 are identified 

4 as peak bins, 474, 476, 478, and 480, respectively, in a Hough space 472. The bin with 

5 the most votes represents the strongest line. 

6 Many other angle finding techniques can be used to find the angle of the spatial 

7 pattern without departing from the scope of the invention. Further still, if the frequency 

8 space is filtered before evaluation of the angle of the spatial pattern(s), the data points on 

9 the frequency space may be altered, such as binarized or represented by edges, for 

10 example. Therefore, the utility and efficiency of any particular angle finding technique 

1 1 may vary depending on the filtering technique used, and/or the representation of the 

12 frequency response in the frequency space, such as by grey level or scaled grey level, for 
^ 1 3 example. 

^ 14 Often more than one spatial pattern is identified within the frequency response. In 

15 a preferred embodiment, the angle of all the spatial patterns are considered in subsequent 

1 6 processes. In another embodiment, the angle of the strongest pattern is used, such as the 

1 7 angle of the projection that produced the highest peak. In another embodiment, the 

18 spatial patterns are weighted, and a dominant pattern is determined from any combination 

19 of the strength, that is number of features along pattern, and/or the weighted value of the 

20 pattern. Weighting is accomplished using one of many methods known in the art, and 

21 aspects of the lines are weighted according to numerous schemes without departing from 

22 the scope of the invention. For instance, a simple weighting scheme is identifying as the 

23 dominant line, the line having a dominant frequency, after the frequency response has 

24 been logarithmically scaled to minimize the effect of the DC peak and other low 

25 frequency peaks. Alternatively, the lines are weighted based on density and/or length, for 

26 example. 

27 The invention recognizes that a spatial pattern, specifically a line, within the 

28 frequency space has an angle in the frequency space that is equivalent to, or at a constant 

29 offset from, the angle of an object and/or features in the image. 
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1 For example, the angle of at least one of the lines in the frequency space will 

2 equal the angle of the length of the leads of a leaded object, as can be seen by examining 

3 Fig. 5B, previously described herein. However, depending on how the angle of the 

4 leaded object is defined, the angle of the lines in the frequency space will be positioned 0° 

5 or 90° offset from the angle of the leaded object For instance, if the angle of the leaded 

6 object is defined as angle 908 depicted in Fig. 9, the angle of the lines in the frequency 

7 space are offset 0° from the angle of the leaded object 900, and if the angle of the leaded 

8 object is defined as being along the horizontal axis, the angle of the lines in the frequency 

9 space are offset 90° from the angle of the leaded object. 

10 Depending on the object or features, the angle does not indicate the direction of 

1 1 the object. For example, the angle, derived as herein described, for the lead sets of Fig 

12 5B, previously described, is 130° and 310°. 

13 Accurate and quick angle information is obtained particularly for objects that 

14 have features that repeat or for repeated objects positioned at the same angle, which is 

15 advantageous in many post-processing applications. Turning to Fig. 6, not drawn to 

16 scale, examples are certain calibration targets 602 with repeating features, pads on a 

17 circuit board 604, liquid crystal displays 610, a group of crescent bonds 606, a group of 

18 wires in a bonding application 608, two-dimensional ID codes 614, leaded objects, such 

19 as surface mount and through hole components, and ball grid arrays, for example. 

20 It should be apparent that angle of other objects without repeating features, are 

21 also amendable to being evaluated, such as a bar code 612, some of the calibration targets 

22 602, or writing 702, shown in Fig. 7, described further hereinafter, for example. Another 

23 example is depicted in Fig. 5D, previously described, which shows an integrated circuit 

24 board, having several features or portions thereof aligned at approximately 30° and 120°, 

25 not drawn to scale. The angle of approximately 30° and 120° is ascertained even though 

26 the features are not uniform or regularly spaced, as in the case of leads, and even though 

27 each feature does not contribute equally to the angle (some features contribute edges 

28 from both sides of the features, while others features contribute edges only from one 

29 side). 
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1 In general, many objects that have at least portions of the object and/or features 

2 oriented in a general direction can be analyzed using the method of Fig, 2. 

3 Typically, determining the angle of the object in the image does not complete the 

4 image analysis. The angle, optionally, becomes input into subsequent image processing 

5 or analysis (206). It should be apparent that any type of post-processing can be 

6 implemented using the angle derived as described herein without departing from the 

7 scope of the invention. The angle can also be one of several desired outputs instead of, or 

8 in addition to, being input for further processing. 

9 Knowing the angle of the features and/or object removes one variable from a 

10 vision task. Now, instead of performing a vision task where attention is paid to 

1 1 translation, rotation, and scale, only variations in translation and scale need to be 

12 accommodated. For instance, searches are often performed by making a model of the 

13 feature to be found, and searching the image at a plurality of angles and positions for a 

14 match of the model to a portion of the image. If scale is variable, this process is repeated 

15 at various scales. However, if the angle is predetermined, the search is executed more 

16 quickly because all possible angles of the model do not need to be examined. For 

17 example, if the vision task was character recognition of a freshness date 702 on an 

18 individual pack of peanuts 700 depicted in Fig. 7. The angle, computed as previously 

19 described, is 45° and/or 135°. Thereafter, the search model is not tested against the 

20 image at other angles, decreasing the search time, and improving time of performance of 

21 the character recognition. 

22 It should be apparent that the angle information can be leveraged in any location, 

23 gauging, inspecting, training, or modeling procedure, for example. For instance, Figs. 

24 10A and Fig. 10B are flowcharts summarizing operation of an embodiment of a method 

25 according to the invention that uses the angle(s), derived as previously described, to 

26 evaluate the image, where steps of the method are denoted in the detailed description in 

27 parentheses. In one embodiment depicted in Fig. 10A, the angle (1000) is leveraged to 

28 segment one or more regions of interest from the image (1002) by using a segmenting 

29 technique that uses, or is quicker given, the angle, where segmenting a region of interest 
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1 of an image is a typical technique used during image processing. One example of a 

2 segmenting technique that uses, and/or is quicker given the angle is using projection 

3 along the angle, or at an offset thereto, to measure the extents of a region along that 

4 angle, or offset thereto. An additional example of a segmenting technique is given 

5 below. Thereafter, the regions of interest and/or the remainder of the image is evaluated 

6 (1004), where again the evaluation can include gauging, inspecting, training, or 

7 modeling, for example. Another embodiment is depicted in Fig. 10B. First, the angle is 

8 provided (1000). Then, one or more regions of interest are optionally segmented, with or 

9 without use of the angle (1006). Next, the regions of interest and/or the remainder of the 

10 image is evaluated by leveraging the angle (1008). Again, the evaluation is gauging, 

1 1 inspecting, training, or modeling, for example, where the evaluation requires, or is made 

12 more efficient, when provided, the angle of the object and/or features in the image. 

13 In a preferred embodiment, the angle is input (800) into a modeling procedure 

14 illustrated by the flowchart of Fig. 8, where steps of the method are denoted in the 

15 detailed description in parentheses. Fig. 8 describes a method that outputs a geometrical 

16 description of an object and/or features of an object given the angle of the feature and/or 

17 object, where the angle is derived as previously described 

18 The angle (800) is leveraged in the segmenting step (802), and/or the gauging 

1 9 portion (806) of the modeling task. 

20 Numerous methods can be used to segment the regions of interest from the image, 

21 with, or without, using the angle. In one embodiment, the regions are segmented (802), 

22 without using the angle information, by determining a threshold grey value that separates 

23 the background from the regions of interest, which is a technique known in the art. As 

24 commonly understood by those trained in the art, determining a threshold to binarize the 

25 image is a difficult task in connectivity analysis. A threshold value that is too large will 

26 remove too much information from the image, and a threshold value that is too small will 

27 retain unwanted information. 

28 Therefore, one technique to segment a region, such as a lead set, chooses a 

29 threshold using an aspect of the regions, such as shape, in conjunction with an iterative 
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1 process, as is further described in co-pending U.S. Application, Serial No. 09/605,441 

2 entitled, "Methods and Apparatuses for Generating from an Image a Model of an 

3 Object," filing date August 28, 200, which is hereby expressly incorporated by reference 

4 in its entirety. For example, the lead set region 914 of Fig, 9A is thresholded from image 

5 940 in the iterative process by checking whether the region(s) segmented at a threshold 

6 value is rectangular. More particularly, pluralities of threshold values are tried, and the 

7 image 940 is binarized using each of the threshold values. The binary images are 

8 examined to determine whether one or more rectangular regions are present, where the 

9 shape is determined using tools known in the art, such as projection, for example. 

10 Different threshold values are tried until the binary image contains a substantially 

1 1 rectangular region or an even number of substantially rectangular regions. 

12 An alternative way to segment the image is using U. S. Patent 5,949,905 Nichani 

13 et al, assigned to Cognex Corporation, incorporated herein by reference. 

14 A preferred embodiment, however, leverages the angle (800) to segment regions 

15 (802), and is described in co-pending U.S. Application, Serial No. 09/457,825, entitled, 

16 "Methods and Apparatuses for Identifying Regions of Similar Texture in an Image," 

17 filing date December 9, 1999, which is hereby expressly incorporated by reference in its 

18 entirety (the "Texture Patent Application"). In this embodiment, the Texture Patent 

19 Application applies a one-dimensional Fourier analysis only at the angles that were 

20 obtained in the previous step (800), as opposed to searching all possible angles. The 

21 regions having frequencies with substantially the highest power are segmented out of the 

22 image, as is described further in the Texture Patent Application. 

23 For example, the angle 45° or 135°, derived as previously described, of the leaded 

24 object 900 in the image 940 is used in accordance with the teachings of the Texture 

25 Patent Application to segment the lead set 914, 

26 When the Texture Patent Application segments (802) the lead set 9 14 from the 

27 image 940, other regions, such as the logo 902, may also be segmented. Therefore, 

28 optionally, extraneous regions are identified and ignored by a post-processing step, such 
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1 as filtering after segmentation (804) or filtering after some (814), or all (818), of the 

2 gauging (806) is complete, 

3 Specifically, regions or features therein having an aspect in conflict with a 

4 validity norm are identified and ignored or removed. For instance, ignoring all non- 

5 rectangular regions will remove from evaluation the region containing the logo 902, 

6 where the shape is determined using tools known in the art, such as a search tool using a 

7 synthetic model, for example. A further example ignores regions above an acceptable 

8 area threshold, given the magnification of the image 940, where the area is determined 

9 using numerous tools known in the art, such as a connectivity tool or a caliper tool, for 

10 instance. 

1 1 The aspects of the regions, and/or features within the regions used to filter the 

12 regions depend on the object being modeled, and, therefore, vary with the application. 

13 As previously described, in a preferred embodiment, the angles of any spatial 

14 pattern found in the frequency space are input to the post processing step. In a preferred 

15 embodiment, the segmenting step indicates whether any of the angles derived as herein 

16 described were not the angles of the object and/or features (i.e., were incorrect angles). 

17 Specifically, no region is segmented using the incorrect angles or no region that passes 

18 the filters is segmented at the incorrect angles. 



19 Next, the object and/or the features of the object are gauged (806), where gauging 

20 can be accomplished using various vision tools, and combinations thereof, tailored to the 

21 object and/or features, as should be apparent to those skilled in the art. 

22 In the leaded object 900 example, each lead set region is gauged. Fig. 9B depicts 

23 a portion of a region 922 of the lead set 914. In one embodiment, the height 924 of the 

24 region 922 provides the lead length 910, and the length 926 of the region 922 provides 

25 the length of the lead set 914 (808). 

26 The dimensions of the leads 904 are measured more accurately as distances 

27 between edges of the leads 904, where the edges are generated by applying an edge 

28 detector (810). From the edge data, the lead width 916, the lead pitch 918, (812) and, 

29 optionally, a refined value for the length of the lead set 914, a refined value for the lead 
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1 length 9 1 0, and/or the foot length 920 (8 1 6) is determined. In one embodiment, edges 

2 corresponding to the sides of the leads 904, such as edge 932 and edge 936, for example, 

3 are found by accumulating edges along a projection axis, /?, of a window 928, where the 

4 window 928 is shaded in Fig. 9B for illustration purposes. 

5 The intensity of pixels within the window 928 along p are projected (i.e., added), 

6 thereby generating a one-dimensional image, which is illustrated graphically as histogram 

7 930 of Fig. 9C. Linear projection collapses an image by summing the grey values of the 

8 pixels in the direction of the projection. The summation tends to amplify edges in the 

9 same direction as/?. After projection, the one-dimensional image is filtered to remove 

10 smaller transitions in grey value. The edge 932 is represented in the histogram 930 as a 

1 1 rising ramp 934, and the edge 936 is represented in the histogram 930 as a falling ramp 

12 938. This functionality (i.e. identifying and filtering the edges) is typically provided in 
CO 13 the industry as a single vision tool, usually known as a caliper tool. Each edge has a 

14 polarity (i.e., direction), where edge 932 has a dark-to-light polarity, also known as a 

^ 1 5 positive polarity, and edge 936 has a light-to-dark polarity, also known as a negative 

°° 1 6 polarity. 

17 For the lead set 914, Fig. 9E> depicts a partial edge list 944 containing the first 

18 seven edges of the histogram 930. The edges are labeled from 0 through 6 according to 

19 the position of the edge from the left to right across the region 922, such that edge 0 is 

20 positioned closest to the beginning of the window 928 and edge 6 edge is the farthest 

21 away of the edges within the partial edge list 944. The edges are also labeled with the 

22 respective polarity by a plus or minus notation placed below the ordinal label. 

23 The location of the regions and the angle of the features and/or object, derived as 

24 previously described, provide the necessary information to properly place the window 

25 928. The width and height of the window 928 are determined using the area of the region 

26 922. 

27 Alternatively, a connectivity analysis, a technique known in the art, segments the 

28 region 922, and the width and the height of the window 928 is determined therefrom, and, 



22 



1 in conjunction with the previously determined angle, the position of the window 928 is 

2 determined therefrom. 

3 Once the edges are generated (810), the lead width 916 and the lead pitch 91 8 is 

4 determined (8 12) using a gauging technique, such as by finding the distance between 

5 parallel edges, which is a technique known in the art. 

6 In one embodiment, the lead width 916 and the lead pitch 918 are found by 

7 pairing, based on polarity, the edges within the edge list 944, so that the edge pairs 

8 represent the sides of each lead 904, and then determining therefrom the lead width 9 12 

9 and the lead pitch 914 (812). 

10 The invention recognizes that pairing using the only known, (i.e., the polarity) 

1 1 will correlate enough of the edges with the proper lead side to determine a representative 

12 lead width and lead pitch, 

13 The polarity constraint is dictated by the lighting of the object during imaging. 



14 For front lighting, the leads 904 image as bright features, and, therefore, an edge pair 

15 corresponding to the first lead 946 in the lead set 914 is a positive edge that is followed in 

16 the edge list 944 by a negative edge of a higher label number A higher label number 

17 indicates the position of the second edge is to the right of the first edge in the edge pair, 
is (i.e. the edges are paired going in the direction from left to right across the region 922 

19 and the histogram 930). Therefore, the edge pair of the first lead 946, using the polarity 

20 constraint, is 0+ and The edge pair corresponding to the second lead 948 is composed 

21 of edges with higher labels than the edge of the first lead 946, and the edge pair includes 

22 a positive edge that is followed in the edge list 944 by a negative edge of a higher label 

23 number (i.e. 3+ and 4-). The edge list 944 is traversed in this manner, from 0 to the last 

24 label (not shown) pairing all possible edges meeting the polarity constraints described 

25 above. Edges that are not paired, such as edge 2-, are, optionally, discarded. 



26 Then, the median width and the median pitch are determined from the positions of 

27 the edges in the edge pairs. 

28 Preferably, the median width and the median pitch are refined using a second set 

29 of edge pairs that are generated by re-traversing the edge list 944, which may include the 
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1 discarded edges, such as edge 2-, and re-pairing the edges using all the known values, 

2 where the known values now include: the polarity, the median width, and the median 

3 pitch. The constraints to pair edges now include both polarity and distance. An edge pair 

4 corresponding to the first lead 946 is a positive edge followed in the edge list 944 by a 

5 negative edge that is positioned at a distance, equal to, or greater than, the median lead 

6 width, plus or minus a tolerance, and has a higher label number, where the value of the 

7 tolerance will depend upon the application. In this example, edge 1- is not within the 

8 tolerance of the median lead width distance from edge 0+. Therefore, the edge pair of the 

9 first lead 946 is 0+ and 2- The first edge of the second lead 948 is again a positive edge, 

10 and it is positioned at a distance, equal to, or greater than, the median lead pitch, plus or 

1 1 minus the tolerance, from the edge 0+ In this example, the first edge of the second lead 

12 948 is edge 3+. The second edge of the second lead 948 is again a negative edge, and it 

13 is positioned at a distance, equal to, or greater than, the median lead pitch, plus or minus 

14 the tolerance, from the edge 2- (i.e., 4-). Alternatively, the second edge of the second 

15 lead 948 is again a negative edge, and it is positioned at a distance, equal to, or greater 

16 than, the median lead width, plus or minus the tolerance, from the edge 3+ (i.e., 4-). The 
n edge list 944 is traversed in this manner from 0 to the last label (not shown) pairing all 

18 possible edges meeting the polarity and distance constraints described above. 

19 Thereafter, the median widths and pitches are recomputed using the refined set of 

20 edge pairs. 

21 For back-lit images, the polarity constraints are reversed, where the term front lit 

22 and back-lit images are known in the art. 

23 The technique described to associate the proper edge with a lead side can be used 

24 to associate edges with other boundaries without departing from the scope of the 

25 invention, such as associated edges with bar codes for instance. It should also be 

26 apparent that when the lighting is known (i.e., the polarity is known) the above technique 

27 can be used to differentiate between edges of a feature of interest and extraneous edges, 

28 whether the extraneous edges, for example, are due to noise or due to features not of 
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1 interest, including differentiating edges from images of ball grid arrays, images of keys in 

2 key chains, and images of numerical key pads on phones or remote controls, for example, 

3 Optionally, the lead set length is refined (816) as the distance between the first 

4 edge 932 of the first lead 946 and the second edge 952 of the last lead 954. 

5 Optionally, the foot length 920 is determined (816) from front-lit images, 

6 assuming that the foot is a single bright region, such as region 528 within Fig. 5B 

7 described previously, for example. A window 942, shaded for illustration purposes, is 

8 positioned, across one or more leads 904, and generally orthogonal to window 928. The 

9 grey values within the window 942 are projected along the projection axis, p 2 , to identify 

10 possible edges. The two edges that surrounding a bright region, that is a positive edge 

1 1 followed by a negative edge going in the up to down direction, and the two edges that are 
% 12 the furthest from the lead body are the edges of the foot 906, which once identified 

^ 13 provide the foot length 920. Optionally, the edges of more than one foot for more than 

±Q 14 one lead 904 can be used to find the median foot length. 

jS 15 Further, when the window 942 crosses the ends of the leads 904 ? optionally, the 

W 16 lead length 920 is also refined (8 1 6) from the edges found therein, 
rj 17 Next, the geometrical description is constructed (820). In addition to the gauged 

,1 18 dimensions, the geometrical description often requires additional values that may be 

19 derived from the gauged dimensional information. In the lead set example, the 

q 20 geometrical description also includes the number of leads within the lead set, where the 

21 number of leads 904 within the lead set 914 equals the distance between the first lead 946 

22 and the last lead 954 divided by the lead pitch 918 plus one. The distance between the 

23 first lead 946 and the last lead 954 is the length 926 of the region 922 or the position of 

24 the last edge 952 minus the position of the first edge 932. Alternatively, the number of 

25 leads 904 equals the number of passing paired edges. The lead length 910, the lead width 

26 916, the lead pitch 918, the number of leads, and/or foot length 920 comprise part, or all, 

27 of a typical geometric description (820) of the lead set 914, which is used as a model. 

28 Additionally, the location 950 and angle 908 comprise, part or all, of the pose of the lead 

29 set 914 in the geometrical description. Further, the angle is, optionally, used to generate 

25 



1 a complete description of a leaded object by properly combining at the angle of the lead 

2 set and the angle of the object the geometrical description of more than one lead set of a 

3 single leaded object 

4 The model is used in vision tasks as is, or refined further. Once the geometrical 

5 description is determined, optionally, features of the object are re-gauged to refine the 

6 dimensions. It should be apparent vision tools known in the art alone or in combination, 

7 can refine part, or all, of the geometrical description once it has been determined as 

8 herein described The lead length and foot length are rougher than the other 

9 measurements, and are easily refined using calipers. In a preferred embodiment, the 

1 0 geometrical description is refined as described in co-pending patent application entitled 



1 1 "Methods and Apparatuses for Refining a Geometric Description of an Object Having a 

Jjf 12 Plurality of Extensions", application Serial No. 09/203, 1 82, filed 1 1/30/98, assigned to 

J 1 13 Cognex Corporation. 

yy 14 It should be apparent that other more sophisticated tools, available in the art, can 

'2 15 be used to further process and/or score edges, known in the art, according to the 

Ul 16 specificity required by the application. 

O 17 It should also be apparent that the gauging description 

,1 18 geometrical description detailed above is an instance of gauging and modeling a gull wing 

■M- 19 lead set, and refinements thereto, or refinements to adapt the gauging and/or construction 

2 20 of the geometrical description to other objects and/or regions can be made without 

21 departing from the scope of the invention. 

22 It should also be apparent that other objects can be modeled, such as a telephone 



23 display 1 100, or an averaged or representative crescent bond 1 102 from among the group 

24 of crescent bonds 1 104 depicted in Fig. 1 1, not drawn to scale. Also, many of the objects 

25 for which the angle can be found, as described herein, can also be usefully modeled, 

26 including but not limited to pads, ball grid arrays, and calibration targets, for example. 

27 Further, any image processing algorithm that uses synthetic models can use a 

28 model generated using the teachings herein once the segmenting, gauging, and/or 

29 constructing aspect of the modeling is tailored to the application. 
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1 Those skilled in the art will appreciate that some, or all, of the steps of generating 

2 a frequency response, finding angles, post-processing, segmenting, filtering, and gauging 

3 described hereinbefore can be combined and effected as hardware implementations, 

4 software implementations, or a combination thereof. 

5 Furthermore, it should be appreciated that any of the images described herein can 

6 be subject to further processing, such as by filtering using a gaussian filter, median filter, 

7 smoothing filter, morphological filter or the like known in the art, in order to improve 

8 image quality. 

9 Although connectivity tools and caliper tools are described herein for obtaining 



10 characteristics relating to the length, width, pitch, and area of the leads, lead sets, and/or 

1 1 feet of the leads for the model generation, it should be appreciated that the information 

12 relating to the characteristics of the features or object can be obtained using other vision 

13 tools known in the art or from a CAD tool or other machinery that requires or acquires 

14 and has information relating to the characteristics of interest Alternatively, portions of 

15 the information can be manually obtained by an operator, entered into the system, and 

16 used in instead of, or as a starting value for, the characteristics. 



17 It should be apparent that the output of the application of the frequency analysis 

18 can be stored, or represented as is, or by using formats other than a frequency image 

19 without departing from the scope of the invention, such as parametric representations to 

20 save space, for example. 

21 It should also be apparent to those skilled in the art that the order of the model 

22 generation can be altered without departing from the scope of the invention. 

23 Those skilled in the art will also realize that using reduced-resolution images upon 

24 which to apply the frequency analysis, evaluate the angle, find and/or evaluate the 

25 regions, or gauge, the image or regions, could decrease processing time. Further, any 

26 combination of full-resolution and reduced-resolution images can be used. However, use 

27 of reduced-resolution images typically results in a loss of accuracy. 

28 Those skilled in the art will realize that processing time can also be decreased by 

29 performing any of the computations described herein using sampled data, such as finding 

27 



1 angle from sampled data, for example, perhaps at the expense of performance. Sampled 

2 data is a subset of the available date points, such as every third date point, for instance. 

3 Those skilled in the art will realize that the frequency analysis can be used to 

4 detect movement over time or changes in angle of features over process steps. 

5 Those skilled in the art will realize that the image can be of an entire object, part 

6 of one object, or part of multiple objects. 

7 Although the invention has been shown and described with respect to exemplary 

8 embodiments thereof, various other changes, omissions, and additions in the form of, and 

9 detail thereof, may be made therein without departing from the spirit and scope of the 

10 invention as claimed Accordingly, the above description is not intended to limit the 

1 1 invention except as indicated in the following claims. 
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is claimed is: 

A method for determining an orientation of an object, the method comprising: 
acquiring an intensity image of the object; 

generating at least a portion of a two-dimensional frequency response of the 
intensity image; 

representing the at least a portion of a two-dimensional frequency response within 
a frequency space, the at least a portion of a frequency response providing 
features arranged in a spatial-frequency pattern within the frequency 
space; and 

finding an orientation of the spatial-frequency pattern within the frequency space, 

thereby providing the orientation of the object. 
The method of claim 1, wherein the intensity image is of a portion of the object. 
The method of claim 1, wherein generating further includes: 
generating at least a portion of a two-dimensional frequency response of the 

intensity image by applying a frequency analysis tool to the intensity 

image. 

The method of claim 3, wherein generating the at least a portion of a two- 
dimensional frequency response of the intensity image includes: 
generating at least a portion of a magnitude of a two-dimensional discrete Fourier 
transform of the intensity image to provide the at least a portion of a two- 
dimensional frequency response. 
The method of claim 4, wherein the two-dimensional discrete Fourier transform 
includes a two-dimensional discrete fast Fourier transform. 
The method of claim 3, wherein generating the at least a portion of a two- 
dimensional frequency response of the intensity image includes: 
generating at least a portion of a two-dimensional discrete cosine-transform of the 
intensity image to provide the at least a portion of a two-dimensional 
frequency response. 
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1 7. The method of claim 3, wherein generating the at least a portion of a two- 

2 dimensional frequency response of the intensity image includes: 

3 generating at least a portion of a two-dimensional discrete sine-transform of the 

4 intensity image to provide the at least a portion of a two-dimensional 

5 frequency response. 

1 8. The method of claim 3, wherein generating the at least a portion of a two- 

2 dimensional frequency response of the intensity image includes: 

3 generating at least a portion of a two-dimensional z-transform of the intensity 

4 image to provide the at least a portion of a two-dimensional frequency 

5 response. 

1 9. The method of claim 1 , wherein representing further includes; 

2 representing the at least a portion of a two-dimensional frequency response as a 

3 logarithmically scaled frequency response within the frequency space. 

1 10. The method of claim 1 ? wherein the frequency space is a frequency image, and 

2 wherein representing the at least a portion of a two-dimensional frequency 

3 response further includes: 

4 scaling the at least a portion of a two-dimensional frequency response using a 

5 scaling function so as to enhance high frequency responses within the at 

6 least a portion of the two-dimensional frequency response to provide a 

7 scaled frequency response; and 

8 mapping the scale response by gray scale on the frequency image. 

1 11. The method of claim 1 , wherein finding further includes: 

2 applying an angle finding means to the frequency space to provide an angle of the 

3 spatial-frequency pattern. 

1 12. The method of claim 1, wherein finding further includes: 

2 identifying the spatial-frequency pattern within the frequency space; and 

3 finding the orientation of the spatial-frequency pattern. 
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The method of claim 1, wherein the at least a portion of a two-dimensional 
frequency response provides features forming a plurality of spatial-frequency 
patterns within the frequency space, and wherein finding further includes: 
finding the orientation of the plurality of spatial-frequency patterns. 
The method of claim 1, wherein the at least a portion of a two-dimensional 
frequency response provides features forming a plurality of spatial-frequency 
patterns within the frequency space, and wherein finding further includes: 
identifying one dominant spatial-frequency pattern from among the plurality of 

spatial-frequency patterns; and 
finding the orientation of the dominant spatial-frequency pattern. 
The method of claim 14, wherein identifying further includes: 
identifying as the dominant spatial-frequency pattern one spatial-frequency 

pattern of the plurality of spatial-frequency patterns that includes a 

greatest number of the features. 
The method of claim 1, wherein the orientation is at a constant offset from the 
orientation of the spatial-frequency pattern. 

The method of claim 16, wherein the constant offset substantially equals zero. 
The method of claim 16, wherein the orientation of the object is defined by an 
orientation angle of a feature on the object. 

The method of claim 1, wherein the spatial-frequency pattern includes a line, the 
line having a line angle, and wherein the orientation of the object is an object 
angle, the object angle having a constant offset from the line angle. 
The method of claim 19, wherein the object is a leaded object having leads, and 
wherein the orientation of the leaded object is defined by an orientation angle of 
one of the leads. 

The method of claim 19, wherein the object angle substantially equals an 
orientation angle of a surface mount object from an axis of the intensity image. 
The method of claim 1, further comprising: 



31 



2 inputting the orientation of the object into a subsequent image processing 

3 algorithm. 

1 23 . A method for determining an orientation of an object, the method comprising: 

2 acquiring an intensity image of the object; 

3 generating at least a portion of at least a two-dimensional frequency response of 

4 the intensity image, frequencies within the at least a portion of at least a 

5 two-dimensional frequency response forming a spatial-frequency pattern; 

6 finding an orientation of the spatial-frequency pattern, thereby providing the 

7 orientation of the object. 

1 24. The method of claim 23, wherein generating further includes: 

2 representing the at least portion of at least a two-dimensional frequency response 
% 3 within a frequency space, features within the frequency space forming the 
P 1 4 spatial-frequency pattern. 

~H l 25. The method of claim 24, wherein the frequency space is a frequency image, and 

2 wherein representing the at least a portion of at least a two-dimensional frequency 

W 3 response further includes: 

■q 4 scaling the at least a portion of at least a two-dimensional frequency response 

5 using a scaling function so as to enhance high frequency responses within 

H 6 the at least a portion of the two-dimensional frequency response to provide 

q 7 a scaled frequency response; and 

S mapping the scale response by gray scale on the frequency image. 

1 26. The method of claim 23 5 wherein generating further includes: 

2 generating at least a portion of at least a two-dimensional frequency response of 

3 the intensity image by applying a frequency analysis tool to the intensity 

4 image. 

1 27. The method of claim 26, wherein generating the at least a portion of at least a 

2 two-dimensional frequency response of the intensity image includes: 
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3 generating at least a portion of a magnitude of at least a two-dimensional discrete 

4 Fourier transform of the intensity image to provide the at least a portion of 

5 at least a two-dimensional frequency response. 

1 28. The method of claim 23, further comprising: 

2 acquiring a plurality of intensity images of the object at different depths within 

3 the object; and 

4 generating a three-dimensional image containing a three-dimensional intensity 

5 representation of the object using the plurality of intensity images of the 

6 object; 

7 wherein generating the at least a portion of at least a two-dimensional frequency 

8 response of the intensity image further includes: 

^ 9 generating at least a portion of a three-dimensional frequency response by 

IP 10 applying a frequency analysis tool to the three-dimensional image, 

yrj l 29. The method of claim 23, wherein the orientation of the object in the intensity 

1 ^ 2 image is at a constant offset from the orientation of the spatial-frequency pattern 

U1 3 in the frequency space. 

l 30. The method of claim 29, wherein the orientation of the object is defined by an 

y f 2 orientation angle of a feature on the object. 

y. i 31. The method of claim 23, wherein the spatial-frequency pattern includes a line, the 

S 2 line having a line angle, and wherein the orientation of the object is an object 

3 angle, the object angle having a constant offset from the line angle. 

1 32. An apparatus for determining an orientation of an object within an intensity 

2 image, the apparatus comprising: 

3 frequency means adapted to generate at least a portion of at least a two- 

4 dimensional frequency response of the intensity image; 

5 a frequency space adapted to receive the at least a portion of at least a two- 

6 dimensional frequency response, the at least a portion of a frequency 

7 response providing features arranged in a spatial-frequency pattern within 

8 the frequency space; and 
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finding means adapted to find an orientation of the spatial-frequency pattern 

within the frequency space, thereby providing the orientation of the object. 
The apparatus of claim 32, wherein the frequency means is further adapted to 
generate at least a portion of at least a two-dimensional frequency response of the 
intensity image by applying a frequency analysis tool to the intensity image. 
The apparatus of claim 33, wherein the frequency means is further adapted to 
generate at least a portion of a magnitude of at least a two-dimensional discrete 
Fourier transform of the intensity image to provide the at least a portion of at least 
a two-dimensional frequency response. 

The apparatus of claim 32, wherein the frequency space is a frequency image, the 
apparatus further comprising: 

scaling means, adapted to scale the at least a portion of at least a two-dimensional 
frequency response on the frequency space using a scaling function so as 
to enhance high frequency responses within the at least a portion of the 
two-dimensional frequency response and to provide a scaled frequency 
response; and 

mapping means, in cooperation with the scaling means, adapted to map the scaled 

frequency response by grey scale on the frequency image. 
The apparatus of claim 32, wherein the orientation of the object in the intensity 
image is at a constant offset from the orientation of the spatial-frequency pattern 
in the frequency space. 

The apparatus of claim 36, wherein the orientation of the object is defined by an 
orientation angle of a feature on the object. 

The apparatus of claim 32, wherein the spatial-frequency pattern includes a line, 
the line having a line angle, and wherein the orientation of the object is an object 
angle, the object angle having a constant offset from the line angle. 
A method for finding an orientation of an object, the method comprising; 
acquiring an intensity image of the object; 
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applying a frequency analysis tool to the intensity image to produce at least a 
portion of a two-dimensional frequency response of the intensity image; 

representing the at least a portion of a two-dimensional frequency response within 
a frequency space, the at least a portion of a frequency response providing 
features arranged in spatial-frequency pattern within the frequency space; 
and 

determining an orientation of the spatial-frequency pattern to provide the 
orientation of the object. 

The method of claim 39, wherein applying the frequency analysis tool includes; 

generating at least a portion of a magnitude of a two-dimensional discrete Fourier 
transform of the intensity image to provide the at least a portion of a two- 
dimensional frequency response. 

The method of claim 39, wherein the frequency space is a frequency image, and 

wherein representing the at least a portion of a two-dimensional frequency 

response fiirther includes: 

scaling the at least a portion of a two-dimensional frequency response using a 
scaling function so as to enhance high frequency responses within the at 
least a portion of the two-dimensional frequency response to provide a 
scaled frequency response; and 

mapping the scale response by gray scale on the frequency image. 

The method of claim 39, wherein the orientation of the object in the intensity 

image is at a constant offset from the orientation of the spatial-frequency pattern 

in the frequency space. 

The method of claim 42, wherein the orientation of the object is defined by an 
orientation angle of a feature on the object. 

The method of claim 39, wherein the spatial-frequency pattern includes a line, the 
line having a line angle, and wherein the orientation of the object is an object 
angle ? the object angle having a constant offset from the line angle. 
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ABSTRACT 

A method and an apparatus are disclosed for finding an orientation of an object 
from the image of the object through the generation of, and subsequent evaluation of, at 
least a portion of a frequency response of the image. The orientation of the object can be 
used in subsequent image processing. 
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1 it should be apparent to one skilled in the art that devices within the centra) 

2 processing unit 1 22 can be partitioned in more than one manner without departing from 

3 the scope of the invention. 

4 Those skilled in the art will appreciate that, in addition to implementation on a 

5 programmable digital data processor, the methods and apparatuses taught herein can be 

6 implemented in special purpose hardware. 

7 Fig. 2 illustrates a flowchart of an embodiment of a method according to the 



8 invention, where steps of the method will be denoted in the detailed description in 

9 parentheses- The first step is to acquire an image of the object (200). Next, a frequency 

10 response of the image, or portion thereof, is generated (202). Thereafter, the frequency 
1 i response, or portion thereof, is analyzed to determine the angle of an object and/or 

_ 12 features in the image (204). 

41 13 The frequency response is analyzed because, as recognized by the invention, the 

pj 1 4 frequency response, or portion thereof, contains information about the orientation of 

Jf! 1 5 objects in the image. More specifically, the invention recognizes that the frequency 

p 16 response, or portion thereof, contains a spatial pattern or patterns that is, or are, related to, 

!" 17 and often the equivalent of, the angle of the object i n the image. 
'f. \ 8 Examples of specific frequency responses of images 300 and 3 1 0 of synthetically 

111 19 generated striped objects, herein after termed idealized object 320 are shown in Fig. 3, 

3 20 where the specific frequency responses 302 and 3 1 2, are a magnitude of the two- 

O 21 dimensional discrete Fourier transform ("2-D DFT") of the images 300 and 310. The 

22 frequency responses 302 and 3 1 2 are represented on a dual-axis frequency space 304 and 

23 314, respectively. 



24 The 2-D DPT is given by the following equations: 

m - 0 n 2 - 0 

26 for0<k|<Nj-l and0<k 2 <N 2 ~1 
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1 X(ki,k 2 ) = 0 for all other ki and k 2 . 

2 X(ki,k 2 ) is the frequency response of the image, ki and k 2 are integers, and 

3 X{ki ? k2X at each k] and k^, is the value of the frequency response in a frequency space 

4 having KI and K2 as frequency axes. The image is a discrete, two-dimensional function, 

5 given by x(ni,n 2 ), where ni and n 2 are integers, and the value of x(ni,n2) at each point 

6 (i.e., each ni andn 2 ) represents a shade of grey in the image. The image size is N* by N 2 , 

7 where Ni and N 2 are positive integers. The image has an y-axis and an x-axis. 



8 The Fourier frequency analysis represents an image, x(ni,n 2 ), of size Ni by N 2 , 

9 (i.e., an Ni x N 2 point sequence), in the frequency domain by X(ki,k 2 ), which is an Ni x 

10 N 2 point sequence. 

1 1 Returning to Fig. 3, image 300, depicts the idealized striped object 320 positioned 



12 in the image 300 so that the dark stripes 326 extend horizontally across the image 300. 

13 At a single y-value, the image 300 does not change grey level along the x-axis. The 

14 frequency response, X(ki,k 2 ), 302, shown on the frequency space 304, reflects the lack of 

15 change in grey level of the striped object 320 along the x-axis. The x-axis of the image 

16 300 contributes only to the DC component of the frequency response. Thus, frequencies 

17 are only present at the DC component along the Kl-axis, specifically frequencies are 

18 present where ki = the DC component Frequencies exist along the K2-axis and are 

19 shown in plot 306, which is a cross-section of the frequency space 304 at AA\ The plot 

20 306 depicts the value of the frequency along the abscissa and the magnitude of the 

21 frequency, Xm, along the ordinate. 



22 As known in the art, X(ki,k 2 ) is often complex and is expressed by a magnitude 

23 component, Xm, and a phase component, 9 X , or a real component, Xr, and a complex 

24 component, Xi. 

25 In a preferred embodiment, the frequency response of the image that is generated 

26 is the magnitude, Xm, where X M - |X(ki,k 2 )| = sqrt(Xr 2 + Xi 2 ). Xm is a measure of the 

27 strength of the spatial frequencies present in the image, and the strength is represented by 

28 the height of the peaks 308 in the plot 306, and is also represented by the brightness or 
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